{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The MIT License (MIT)\n",
    "\n",
    "# Copyright (c) 2020, NVIDIA CORPORATION.\n",
    "\n",
    "# Permission is hereby granted, free of charge, to any person obtaining a copy of\n",
    "# this software and associated documentation files (the \"Software\"), to deal in\n",
    "# the Software without restriction, including without limitation the rights to\n",
    "# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of\n",
    "# the Software, and to permit persons to whom the Software is furnished to do so,\n",
    "# subject to the following conditions:\n",
    "\n",
    "# The above copyright notice and this permission notice shall be included in all\n",
    "# copies or substantial portions of the Software.\n",
    "\n",
    "# THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n",
    "# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS\n",
    "# FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR\n",
    "# COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER\n",
    "# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\n",
    "# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tutorial: Feature Engineering for Recommender Systems\n",
    "\n",
    "# 6. Scaling to Production Systems\n",
    "\n",
    "## 6.1. Introduction to dask and dask_cudf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Theory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Acknowledgement**: Much of the introductory material included here is borrowed from other Dask documentation and tutorials.\n",
    "- [\"Dask Video Tutorial\"](https://github.com/jacobtomlinson/dask-video-tutorial-2020) \n",
    "- [YouTube link](https://www.youtube.com/watch?v=_u0OQm9qf_A)\n",
    "- [Introduction To Dask by Richard (Rick) Zamora](https://github.com/rjzamora/notebooks/tree/master/nvtabular_dask_demo)\n",
    "\n",
    "Other useful Dask resources:\n",
    "- [Dask.org](https://dask.org/)\n",
    "    - [Tutorial pages](https://tutorial.dask.org/00_overview.html)\n",
    "- [GitHub Tutorial](https://github.com/dask/dask-tutorial)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is Dask\n",
    "\n",
    "**Very Short Answer**: Dask is an open-source library designed to natively scale Python code.\n",
    "\n",
    "**Slightly-Longer Short Answer**: Dask is a task-based library for parallel scheduling and execution. Although it is certainly possible to use the task-scheduling machinery directly to implement customized parallel workflows (we do it in NVTabular), most users only interact with Dask through a *Dask Collection API*.  The most popular \"collection\" API's include:\n",
    "\n",
    "- [Dask DataFrame](https://docs.dask.org/en/latest/dataframe.html): Dask-based version of the [Pandas](https://pandas.pydata.org/) DataFrame/Series API.  Note that `dask_cudf` is just a wrapper around this collection module (`dask.dataframe`).\n",
    "- [Dask Array](https://docs.dask.org/en/latest/array.html): Dask-based version of the [NumPy]() array API\n",
    "- [Dask Bag](https://docs.dask.org/en/latest/bag.html): *Similar to* a Dask-based version of PyToolz or a Pythonic version of PySpark RDD\n",
    "\n",
    "\n",
    "For example, Dask DataFrame provides a convenient API for decomposing large pandas (or cuDF) DataFrame/Series objects into a collection of DataFrame *partitions*.  This tutorial will focus mostly on this particular Dask collection (since it is the basis for `dask_cudf`).  However, instead of relying only on the established `dask.dataframe` API, we will also see how it is possible (perhaps easy) to implement a custom task graph to operate on Dask-DataFrame objects when necessary.\n",
    "\n",
    "<img src=\"../images/dask-dataframe.svg\" width=\"350px\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dask Uses DAGs Internally\n",
    "\n",
    "Before we start writing any code, it is useful to understand (on a basic level) how Dask actually works. When an application or library uses a Dask collection API (like Dask DataFrame), they are typically using that API to construct a directed acyclic graph (DAG) of tasks.  Once a DAG is constructed, the **core** Dask API can be used (either directly or implicitly through the collection API) to schedule and execute the DAG on one or more threads/processes.\n",
    "\n",
    "In other words, Dask provides various APIs to:\n",
    "\n",
    "1. Construct a DAG of \"tasks\"\n",
    "2. Schedule/execute those DAGs\n",
    "3. (Optionally) Spin up a dedicated worker and scheduler processes to enable distributed execution\n",
    "\n",
    "<img src='../images/dask_dag_cartoon.png' width=500>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Important Components of the \"Dask Ecosystem\"\n",
    "\n",
    "The components of the Dask ecosystem that are most critical for NVTabular (and will be discussed in this tutorial) are:\n",
    "\n",
    "- `dask` (core Dask library): [[GitHub Repo](https://github.com/dask/dask)]  This is the core Dask library.  It also contains the Dask Dataframe API (`dask.dataframe`)\n",
    "- `dask_cudf`: [[GitHub Repo](https://github.com/rapidsai/cudf/tree/branch-0.15/python/dask_cudf)] This is effectively a wrapper around the `dask.dataframe` module defined in the core Dask library.  Note that a `dask_cudf.DataFrame` object should be thought of as a `dask.dataframe.DataFrame` object, but with the underlying partitions being `cudf.DataFrame`'s (rather than `pandas.DataFrame`)\n",
    "- `distributed`: [[GitHub Repo](https://github.com/dask/distributed)] Distributed version of the Dask execution model (includes the necessary code for scheduling, execution and communication between distributed processes).  This library does not deal with the construction of DAGs, just with the scheduling and execution of DAGs on distributed *workers*.\n",
    "- `dask_cuda`: [[GitHub Repo](https://github.com/rapidsai/dask-cuda)] Provides various utilities to improve deployment and management of distributed Dask *workers* on CUDA-enabled systems."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## HandsOn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we get started, it is convenient to create a simple `dask.distributed` client. If we work with a small dataset, then it is not necessary to initialize a `dask.distributed` client. The code should run in the same way."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import dask\n",
    "from dask.distributed import Client, LocalCluster\n",
    "import dask.dataframe as dd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table style=\"border: 2px solid white;\">\n",
       "<tr>\n",
       "<td style=\"vertical-align: top; border: 0px solid white\">\n",
       "<h3 style=\"text-align: left;\">Client</h3>\n",
       "<ul style=\"text-align: left; list-style: none; margin: 0; padding: 0;\">\n",
       "  <li><b>Scheduler: </b>tcp://127.0.0.1:40757</li>\n",
       "  <li><b>Dashboard: </b><a href='http://127.0.0.1:8787/status' target='_blank'>http://127.0.0.1:8787/status</a></li>\n",
       "</ul>\n",
       "</td>\n",
       "<td style=\"vertical-align: top; border: 0px solid white\">\n",
       "<h3 style=\"text-align: left;\">Cluster</h3>\n",
       "<ul style=\"text-align: left; list-style:none; margin: 0; padding: 0;\">\n",
       "  <li><b>Workers: </b>8</li>\n",
       "  <li><b>Cores: </b>8</li>\n",
       "  <li><b>Memory: </b>132.13 GB</li>\n",
       "</ul>\n",
       "</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<Client: 'tcp://127.0.0.1:40757' processes=8 threads=8, memory=132.13 GB>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "client = Client(n_workers=8, \n",
    "                threads_per_worker=1,\n",
    "                memory_limit='50GB',\n",
    "                ip='127.0.0.1')\n",
    "client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 28 ms, sys: 0 ns, total: 28 ms\n",
      "Wall time: 27.4 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "ddf_train = dd.read_parquet('../data/train.parquet', blocksize=12e3)\n",
    "ddf_valid = dd.read_parquet('../data/valid.parquet', blocksize=12e3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><strong>Dask DataFrame Structure:</strong></div>\n",
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event_time</th>\n",
       "      <th>event_type</th>\n",
       "      <th>product_id</th>\n",
       "      <th>brand</th>\n",
       "      <th>price</th>\n",
       "      <th>user_id</th>\n",
       "      <th>user_session</th>\n",
       "      <th>target</th>\n",
       "      <th>cat_0</th>\n",
       "      <th>cat_1</th>\n",
       "      <th>cat_2</th>\n",
       "      <th>cat_3</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>ts_hour</th>\n",
       "      <th>ts_minute</th>\n",
       "      <th>ts_weekday</th>\n",
       "      <th>ts_day</th>\n",
       "      <th>ts_month</th>\n",
       "      <th>ts_year</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>npartitions=1</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <td>object</td>\n",
       "      <td>object</td>\n",
       "      <td>int64</td>\n",
       "      <td>object</td>\n",
       "      <td>float64</td>\n",
       "      <td>int64</td>\n",
       "      <td>object</td>\n",
       "      <td>int64</td>\n",
       "      <td>object</td>\n",
       "      <td>object</td>\n",
       "      <td>object</td>\n",
       "      <td>object</td>\n",
       "      <td>object</td>\n",
       "      <td>int64</td>\n",
       "      <td>int64</td>\n",
       "      <td>int64</td>\n",
       "      <td>int64</td>\n",
       "      <td>int64</td>\n",
       "      <td>int64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "<div>Dask Name: read-parquet, 1 tasks</div>"
      ],
      "text/plain": [
       "Dask DataFrame Structure:\n",
       "              event_time event_type product_id   brand    price user_id user_session target   cat_0   cat_1   cat_2   cat_3 timestamp ts_hour ts_minute ts_weekday ts_day ts_month ts_year\n",
       "npartitions=1                                                                                                                                                                             \n",
       "                  object     object      int64  object  float64   int64       object  int64  object  object  object  object    object   int64     int64      int64  int64    int64   int64\n",
       "                     ...        ...        ...     ...      ...     ...          ...    ...     ...     ...     ...     ...       ...     ...       ...        ...    ...      ...     ...\n",
       "Dask Name: read-parquet, 1 tasks"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ddf_train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we have created a `dask.dataframe.DataFrame` object called `ddf_train` and `ddf_valid`.  Both are essentially a (**lazy**) collection of pandas dataframes. Dask loaded the metadata (DataFrame schema) but did not load any data in-memory. Each pandas dataframe in this collection is called a **partition**.  We can access this property (the total number of partitions) using the `DataFrame.npartitions` attribute.\n",
    "\n",
    "**It is absolutely critical to recognize that `ddf_train` and `ddf_valid` are *not* actually backed by *in-memory* pandas data, but instead by a DAG of tasks**.  This DAG (accessible via `ddf.dask`) specifies the exact network of operations needed to produce the underlying partitions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event_time</th>\n",
       "      <th>event_type</th>\n",
       "      <th>product_id</th>\n",
       "      <th>brand</th>\n",
       "      <th>price</th>\n",
       "      <th>user_id</th>\n",
       "      <th>user_session</th>\n",
       "      <th>target</th>\n",
       "      <th>cat_0</th>\n",
       "      <th>cat_1</th>\n",
       "      <th>cat_2</th>\n",
       "      <th>cat_3</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>ts_hour</th>\n",
       "      <th>ts_minute</th>\n",
       "      <th>ts_weekday</th>\n",
       "      <th>ts_day</th>\n",
       "      <th>ts_month</th>\n",
       "      <th>ts_year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [event_time, event_type, product_id, brand, price, user_id, user_session, target, cat_0, cat_1, cat_2, cat_3, timestamp, ts_hour, ts_minute, ts_weekday, ts_day, ts_month, ts_year]\n",
       "Index: []"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ddf_train._meta"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's work on some examples: Simplified Target Encoding\n",
    "1. We combine two columns cat_2 and brand \n",
    "2. We TargetEncode the new column cat_2_brand \n",
    "3. We merge the counts back to the train and validation dataset\n",
    "4. We overwrite counts with less than 20 for on cat_2_brand with global_mean"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the execution time is 117ms - meaning that dask has registered the operations but hasn't executed them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 60 ms, sys: 0 ns, total: 60 ms\n",
      "Wall time: 56.7 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "ddf_train['cat_2_brand'] = ddf_train['cat_2'].astype(str) + '_' + ddf_train['brand'].astype(str)\n",
    "ddf_valid['cat_2_brand'] = ddf_valid['cat_2'].astype(str) + '_' + ddf_valid['brand'].astype(str)\n",
    "\n",
    "ddf_train_group = ddf_train[['cat_2_brand', 'target']].groupby(['cat_2_brand']).agg(['count', 'mean'])\n",
    "ddf_train_group = ddf_train_group.reset_index()\n",
    "ddf_train_group.columns = ['cat_2_brand', 'TE_count', 'TE_mean']\n",
    "ddf_train = ddf_train.merge(ddf_train_group, how='left', on='cat_2_brand')\n",
    "ddf_valid = ddf_valid.merge(ddf_train_group, how='left', on='cat_2_brand')\n",
    "global_mean = ddf_train['target'].mean()\n",
    "ddf_train['TE_mean'] = ddf_train.TE_mean.where(ddf_train['TE_count']>20, global_mean)\n",
    "ddf_valid['TE_mean'] = ddf_valid.TE_mean.where(ddf_valid['TE_count']>20, global_mean)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can compute the task graph by calling `.compute()` or `.persist()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 15.1 s, sys: 9.68 s, total: 24.8 s\n",
      "Wall time: 3min 14s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event_time</th>\n",
       "      <th>event_type</th>\n",
       "      <th>product_id</th>\n",
       "      <th>brand</th>\n",
       "      <th>price</th>\n",
       "      <th>user_id</th>\n",
       "      <th>user_session</th>\n",
       "      <th>target</th>\n",
       "      <th>cat_0</th>\n",
       "      <th>cat_1</th>\n",
       "      <th>...</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>ts_hour</th>\n",
       "      <th>ts_minute</th>\n",
       "      <th>ts_weekday</th>\n",
       "      <th>ts_day</th>\n",
       "      <th>ts_month</th>\n",
       "      <th>ts_year</th>\n",
       "      <th>cat_2_brand</th>\n",
       "      <th>TE_count</th>\n",
       "      <th>TE_mean</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2020-03-01 00:00:59 UTC</td>\n",
       "      <td>cart</td>\n",
       "      <td>6902464</td>\n",
       "      <td>zlatek</td>\n",
       "      <td>49.91</td>\n",
       "      <td>531574188</td>\n",
       "      <td>48714293-b3f9-4946-8135-eb1ea05ead74</td>\n",
       "      <td>0</td>\n",
       "      <td>electronics</td>\n",
       "      <td>telephone</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-01 00:00:59</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>nan_zlatek</td>\n",
       "      <td>607.0</td>\n",
       "      <td>0.258649</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2020-03-01 00:01:20 UTC</td>\n",
       "      <td>cart</td>\n",
       "      <td>1002544</td>\n",
       "      <td>apple</td>\n",
       "      <td>397.10</td>\n",
       "      <td>622090790</td>\n",
       "      <td>fb5b918c-f1f6-48d9-bcf4-7eb46e83fc6b</td>\n",
       "      <td>0</td>\n",
       "      <td>construction</td>\n",
       "      <td>tools</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-01 00:01:20</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>light_apple</td>\n",
       "      <td>1013391.0</td>\n",
       "      <td>0.469441</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2020-03-01 00:01:52 UTC</td>\n",
       "      <td>cart</td>\n",
       "      <td>1003316</td>\n",
       "      <td>apple</td>\n",
       "      <td>823.70</td>\n",
       "      <td>622090543</td>\n",
       "      <td>b821ee79-96fe-4979-be9d-21ee2e6777c3</td>\n",
       "      <td>0</td>\n",
       "      <td>construction</td>\n",
       "      <td>tools</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-01 00:01:52</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>light_apple</td>\n",
       "      <td>1013391.0</td>\n",
       "      <td>0.469441</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2020-03-01 00:02:14 UTC</td>\n",
       "      <td>cart</td>\n",
       "      <td>16600067</td>\n",
       "      <td>rivertoys</td>\n",
       "      <td>422.15</td>\n",
       "      <td>616437533</td>\n",
       "      <td>aad023bc-c858-47ab-a3a7-ff4654f11b9a</td>\n",
       "      <td>0</td>\n",
       "      <td>sport</td>\n",
       "      <td>trainer</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-01 00:02:14</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>nan_rivertoys</td>\n",
       "      <td>10564.0</td>\n",
       "      <td>0.104411</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2020-03-01 00:02:15 UTC</td>\n",
       "      <td>cart</td>\n",
       "      <td>3701428</td>\n",
       "      <td>arnica</td>\n",
       "      <td>69.24</td>\n",
       "      <td>516454226</td>\n",
       "      <td>ee22b80c-ed3e-3c83-d397-fb69a44d4864</td>\n",
       "      <td>0</td>\n",
       "      <td>appliances</td>\n",
       "      <td>environment</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-01 00:02:15</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>vacuum_arnica</td>\n",
       "      <td>4450.0</td>\n",
       "      <td>0.325393</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461714</th>\n",
       "      <td>2020-03-31 23:57:47 UTC</td>\n",
       "      <td>purchase</td>\n",
       "      <td>24100293</td>\n",
       "      <td>cocochoco</td>\n",
       "      <td>2.65</td>\n",
       "      <td>513094047</td>\n",
       "      <td>d27f822c-f707-4956-a6c3-4ad8fec00cc7</td>\n",
       "      <td>1</td>\n",
       "      <td>appliances</td>\n",
       "      <td>personal</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-31 23:57:47</td>\n",
       "      <td>23</td>\n",
       "      <td>57</td>\n",
       "      <td>1</td>\n",
       "      <td>31</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>massager_cocochoco</td>\n",
       "      <td>82.0</td>\n",
       "      <td>0.146341</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461715</th>\n",
       "      <td>2020-03-31 23:58:19 UTC</td>\n",
       "      <td>purchase</td>\n",
       "      <td>100049773</td>\n",
       "      <td>None</td>\n",
       "      <td>234.96</td>\n",
       "      <td>620580925</td>\n",
       "      <td>c33fde42-a5de-4a1f-9e1c-2ac7518a7d41</td>\n",
       "      <td>1</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-31 23:58:19</td>\n",
       "      <td>23</td>\n",
       "      <td>58</td>\n",
       "      <td>1</td>\n",
       "      <td>31</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>nan_nan</td>\n",
       "      <td>521515.0</td>\n",
       "      <td>0.277106</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461716</th>\n",
       "      <td>2020-03-31 23:58:20 UTC</td>\n",
       "      <td>purchase</td>\n",
       "      <td>3700689</td>\n",
       "      <td>samsung</td>\n",
       "      <td>223.92</td>\n",
       "      <td>514905289</td>\n",
       "      <td>e40783c5-7b21-429f-99af-539d2842e6d3</td>\n",
       "      <td>1</td>\n",
       "      <td>appliances</td>\n",
       "      <td>environment</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-31 23:58:20</td>\n",
       "      <td>23</td>\n",
       "      <td>58</td>\n",
       "      <td>1</td>\n",
       "      <td>31</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>vacuum_samsung</td>\n",
       "      <td>68239.0</td>\n",
       "      <td>0.392459</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461717</th>\n",
       "      <td>2020-03-31 23:59:19 UTC</td>\n",
       "      <td>purchase</td>\n",
       "      <td>100077607</td>\n",
       "      <td>vitek</td>\n",
       "      <td>100.36</td>\n",
       "      <td>633281427</td>\n",
       "      <td>667a8535-221c-4169-aab4-a1972610f102</td>\n",
       "      <td>1</td>\n",
       "      <td>appliances</td>\n",
       "      <td>environment</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-31 23:59:19</td>\n",
       "      <td>23</td>\n",
       "      <td>59</td>\n",
       "      <td>1</td>\n",
       "      <td>31</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>vacuum_vitek</td>\n",
       "      <td>11950.0</td>\n",
       "      <td>0.340000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461718</th>\n",
       "      <td>2020-03-31 23:59:27 UTC</td>\n",
       "      <td>purchase</td>\n",
       "      <td>100068493</td>\n",
       "      <td>samsung</td>\n",
       "      <td>319.41</td>\n",
       "      <td>635165435</td>\n",
       "      <td>861f2378-076f-4ddd-85e3-9844923d03a9</td>\n",
       "      <td>1</td>\n",
       "      <td>construction</td>\n",
       "      <td>tools</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-31 23:59:27</td>\n",
       "      <td>23</td>\n",
       "      <td>59</td>\n",
       "      <td>1</td>\n",
       "      <td>31</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>light_samsung</td>\n",
       "      <td>1212393.0</td>\n",
       "      <td>0.481047</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2461719 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                      event_time event_type  product_id      brand   price  \\\n",
       "0        2020-03-01 00:00:59 UTC       cart     6902464     zlatek   49.91   \n",
       "1        2020-03-01 00:01:20 UTC       cart     1002544      apple  397.10   \n",
       "2        2020-03-01 00:01:52 UTC       cart     1003316      apple  823.70   \n",
       "3        2020-03-01 00:02:14 UTC       cart    16600067  rivertoys  422.15   \n",
       "4        2020-03-01 00:02:15 UTC       cart     3701428     arnica   69.24   \n",
       "...                          ...        ...         ...        ...     ...   \n",
       "2461714  2020-03-31 23:57:47 UTC   purchase    24100293  cocochoco    2.65   \n",
       "2461715  2020-03-31 23:58:19 UTC   purchase   100049773       None  234.96   \n",
       "2461716  2020-03-31 23:58:20 UTC   purchase     3700689    samsung  223.92   \n",
       "2461717  2020-03-31 23:59:19 UTC   purchase   100077607      vitek  100.36   \n",
       "2461718  2020-03-31 23:59:27 UTC   purchase   100068493    samsung  319.41   \n",
       "\n",
       "           user_id                          user_session  target  \\\n",
       "0        531574188  48714293-b3f9-4946-8135-eb1ea05ead74       0   \n",
       "1        622090790  fb5b918c-f1f6-48d9-bcf4-7eb46e83fc6b       0   \n",
       "2        622090543  b821ee79-96fe-4979-be9d-21ee2e6777c3       0   \n",
       "3        616437533  aad023bc-c858-47ab-a3a7-ff4654f11b9a       0   \n",
       "4        516454226  ee22b80c-ed3e-3c83-d397-fb69a44d4864       0   \n",
       "...            ...                                   ...     ...   \n",
       "2461714  513094047  d27f822c-f707-4956-a6c3-4ad8fec00cc7       1   \n",
       "2461715  620580925  c33fde42-a5de-4a1f-9e1c-2ac7518a7d41       1   \n",
       "2461716  514905289  e40783c5-7b21-429f-99af-539d2842e6d3       1   \n",
       "2461717  633281427  667a8535-221c-4169-aab4-a1972610f102       1   \n",
       "2461718  635165435  861f2378-076f-4ddd-85e3-9844923d03a9       1   \n",
       "\n",
       "                cat_0        cat_1  ...            timestamp ts_hour  \\\n",
       "0         electronics    telephone  ...  2020-03-01 00:00:59       0   \n",
       "1        construction        tools  ...  2020-03-01 00:01:20       0   \n",
       "2        construction        tools  ...  2020-03-01 00:01:52       0   \n",
       "3               sport      trainer  ...  2020-03-01 00:02:14       0   \n",
       "4          appliances  environment  ...  2020-03-01 00:02:15       0   \n",
       "...               ...          ...  ...                  ...     ...   \n",
       "2461714    appliances     personal  ...  2020-03-31 23:57:47      23   \n",
       "2461715          None         None  ...  2020-03-31 23:58:19      23   \n",
       "2461716    appliances  environment  ...  2020-03-31 23:58:20      23   \n",
       "2461717    appliances  environment  ...  2020-03-31 23:59:19      23   \n",
       "2461718  construction        tools  ...  2020-03-31 23:59:27      23   \n",
       "\n",
       "        ts_minute  ts_weekday  ts_day  ts_month  ts_year         cat_2_brand  \\\n",
       "0               0           6       1         3     2020          nan_zlatek   \n",
       "1               1           6       1         3     2020         light_apple   \n",
       "2               1           6       1         3     2020         light_apple   \n",
       "3               2           6       1         3     2020       nan_rivertoys   \n",
       "4               2           6       1         3     2020       vacuum_arnica   \n",
       "...           ...         ...     ...       ...      ...                 ...   \n",
       "2461714        57           1      31         3     2020  massager_cocochoco   \n",
       "2461715        58           1      31         3     2020             nan_nan   \n",
       "2461716        58           1      31         3     2020      vacuum_samsung   \n",
       "2461717        59           1      31         3     2020        vacuum_vitek   \n",
       "2461718        59           1      31         3     2020       light_samsung   \n",
       "\n",
       "          TE_count   TE_mean  \n",
       "0            607.0  0.258649  \n",
       "1        1013391.0  0.469441  \n",
       "2        1013391.0  0.469441  \n",
       "3          10564.0  0.104411  \n",
       "4           4450.0  0.325393  \n",
       "...            ...       ...  \n",
       "2461714       82.0  0.146341  \n",
       "2461715   521515.0  0.277106  \n",
       "2461716    68239.0  0.392459  \n",
       "2461717    11950.0  0.340000  \n",
       "2461718  1212393.0  0.481047  \n",
       "\n",
       "[2461719 rows x 22 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "ddf_train.compute()\n",
    "ddf_valid.compute()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "client.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**About `compute`**: The `compute` method is [defined for all Dask collections](https://github.com/dask/dask/blob/51d3f1120fc55f21b5ce1ac137201ea01d9cf496/dask/base.py#L143). For Dask DataFrame, this method will (1) trigger the execution of the graph and (2) convert the Dask DataFrame into a **single** Pandas DataFrame. *This means that you should be sure the pandas equivalent of `ddf` will fit in memory before you use `compute`!*\n",
    "\n",
    "__Using `persist`__\n",
    "\n",
    "Since the `compute` method will convert your Dask DataFrame to a Pandas DataFrame, it is typically a **bad** idea to use compute on larger-than-memory (LTM) datasets.  In NVTabular, we do use a `compute` method, but never on a full Dask/dask_cudf DataFrame object.  Instead, we use `compute` to trigger the collection/reduction of an aggregated statistics dictionary, and/or to write out a processed dataset.\n",
    "\n",
    "In order to execute the `ddf` DAG **without** converting it to a single pandas DataFrame, you need to use the [`persist` method](https://github.com/dask/dask/blob/51d3f1120fc55f21b5ce1ac137201ea01d9cf496/dask/base.py#L101). This function is particularly useful when using distributed systems, because the results will be kept in distributed memory, rather than returned to the local process as with compute. It will also allow the distributed cluster to clean up data that the scheduler no longer deems necessary.  For the single-machine case, the method is used less often."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Let's move on to the GPU accelerated version with dask_cudf.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use `nvidia-smi` command to check the usage of our GPU."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mon Sep 21 14:10:27 2020       \n",
      "+-----------------------------------------------------------------------------+\n",
      "| NVIDIA-SMI 440.64.00    Driver Version: 440.64.00    CUDA Version: 10.2     |\n",
      "|-------------------------------+----------------------+----------------------+\n",
      "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
      "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
      "|===============================+======================+======================|\n",
      "|   0  Tesla T4            Off  | 00000000:00:1E.0 Off |                    0 |\n",
      "| N/A   31C    P8     9W /  70W |      0MiB / 15109MiB |      0%      Default |\n",
      "+-------------------------------+----------------------+----------------------+\n",
      "                                                                               \n",
      "+-----------------------------------------------------------------------------+\n",
      "| Processes:                                                       GPU Memory |\n",
      "|  GPU       PID   Type   Process name                             Usage      |\n",
      "|=============================================================================|\n",
      "|  No running processes found                                                 |\n",
      "+-----------------------------------------------------------------------------+\n"
     ]
    }
   ],
   "source": [
    "!nvidia-smi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import dask as dask, dask_cudf\n",
    "from dask.distributed import Client\n",
    "from dask_cuda import LocalCUDACluster"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table style=\"border: 2px solid white;\">\n",
       "<tr>\n",
       "<td style=\"vertical-align: top; border: 0px solid white\">\n",
       "<h3 style=\"text-align: left;\">Client</h3>\n",
       "<ul style=\"text-align: left; list-style: none; margin: 0; padding: 0;\">\n",
       "  <li><b>Scheduler: </b>tcp://127.0.0.1:34691</li>\n",
       "  <li><b>Dashboard: </b><a href='http://127.0.0.1:8787/status' target='_blank'>http://127.0.0.1:8787/status</a></li>\n",
       "</ul>\n",
       "</td>\n",
       "<td style=\"vertical-align: top; border: 0px solid white\">\n",
       "<h3 style=\"text-align: left;\">Cluster</h3>\n",
       "<ul style=\"text-align: left; list-style:none; margin: 0; padding: 0;\">\n",
       "  <li><b>Workers: </b>1</li>\n",
       "  <li><b>Cores: </b>1</li>\n",
       "  <li><b>Memory: </b>16.52 GB</li>\n",
       "</ul>\n",
       "</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<Client: 'tcp://127.0.0.1:34691' processes=1 threads=1, memory=16.52 GB>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cluster = LocalCUDACluster(ip='127.0.0.1',\n",
    "                           rmm_pool_size=\"16GB\")\n",
    "client = Client(cluster)\n",
    "client"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We reserve 14GB per GPU via `rmm_pool_size`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mon Sep 21 14:10:33 2020       \n",
      "+-----------------------------------------------------------------------------+\n",
      "| NVIDIA-SMI 440.64.00    Driver Version: 440.64.00    CUDA Version: 10.2     |\n",
      "|-------------------------------+----------------------+----------------------+\n",
      "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
      "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
      "|===============================+======================+======================|\n",
      "|   0  Tesla T4            Off  | 00000000:00:1E.0 Off |                    0 |\n",
      "| N/A   33C    P0    26W /  70W |    605MiB / 15109MiB |      0%      Default |\n",
      "+-------------------------------+----------------------+----------------------+\n",
      "                                                                               \n",
      "+-----------------------------------------------------------------------------+\n",
      "| Processes:                                                       GPU Memory |\n",
      "|  GPU       PID   Type   Process name                             Usage      |\n",
      "|=============================================================================|\n",
      "+-----------------------------------------------------------------------------+\n"
     ]
    }
   ],
   "source": [
    "!nvidia-smi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use `dask_cudf` to read and load the data. The remaining code is exactly the same as the dask pandas version."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 996 ms, sys: 416 ms, total: 1.41 s\n",
      "Wall time: 1.41 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "ddf_train = dask_cudf.read_parquet('../data/train.parquet')\n",
    "ddf_valid = dask_cudf.read_parquet('../data/valid.parquet')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 544 ms, sys: 0 ns, total: 544 ms\n",
      "Wall time: 652 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "ddf_train['cat_2_brand'] = ddf_train['cat_2'].astype(str) + '_' + ddf_train['brand'].astype(str)\n",
    "ddf_valid['cat_2_brand'] = ddf_valid['cat_2'].astype(str) + '_' + ddf_valid['brand'].astype(str)\n",
    "\n",
    "ddf_train_group = ddf_train[['cat_2_brand', 'target']].groupby(['cat_2_brand']).agg(['count', 'mean'])\n",
    "ddf_train_group = ddf_train_group.reset_index()\n",
    "ddf_train_group.columns = ['cat_2_brand', 'TE_count', 'TE_mean']\n",
    "ddf_train = ddf_train.merge(ddf_train_group, how='left', on='cat_2_brand')\n",
    "ddf_valid = ddf_valid.merge(ddf_train_group, how='left', on='cat_2_brand')\n",
    "global_mean = ddf_train['target'].mean()\n",
    "ddf_train['TE_mean'] = ddf_train.TE_mean.where(ddf_train['TE_count']>20, global_mean)\n",
    "ddf_valid['TE_mean'] = ddf_valid.TE_mean.where(ddf_valid['TE_count']>20, global_mean)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 2.75 s, sys: 5.77 s, total: 8.52 s\n",
      "Wall time: 14.3 s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event_time</th>\n",
       "      <th>event_type</th>\n",
       "      <th>product_id</th>\n",
       "      <th>brand</th>\n",
       "      <th>price</th>\n",
       "      <th>user_id</th>\n",
       "      <th>user_session</th>\n",
       "      <th>target</th>\n",
       "      <th>cat_0</th>\n",
       "      <th>cat_1</th>\n",
       "      <th>...</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>ts_hour</th>\n",
       "      <th>ts_minute</th>\n",
       "      <th>ts_weekday</th>\n",
       "      <th>ts_day</th>\n",
       "      <th>ts_month</th>\n",
       "      <th>ts_year</th>\n",
       "      <th>cat_2_brand</th>\n",
       "      <th>TE_count</th>\n",
       "      <th>TE_mean</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2020-03-01 08:16:04 UTC</td>\n",
       "      <td>cart</td>\n",
       "      <td>1005135</td>\n",
       "      <td>apple</td>\n",
       "      <td>1516.10</td>\n",
       "      <td>620967403</td>\n",
       "      <td>2f69a6e0-3a9e-4b7c-b717-ce5b8ad85ce3</td>\n",
       "      <td>0</td>\n",
       "      <td>construction</td>\n",
       "      <td>tools</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-01 08:16:04</td>\n",
       "      <td>8</td>\n",
       "      <td>16</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>light_apple</td>\n",
       "      <td>1013391</td>\n",
       "      <td>0.469441</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2020-03-01 08:16:08 UTC</td>\n",
       "      <td>cart</td>\n",
       "      <td>1005135</td>\n",
       "      <td>apple</td>\n",
       "      <td>1516.10</td>\n",
       "      <td>620967403</td>\n",
       "      <td>2f69a6e0-3a9e-4b7c-b717-ce5b8ad85ce3</td>\n",
       "      <td>0</td>\n",
       "      <td>construction</td>\n",
       "      <td>tools</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-01 08:16:08</td>\n",
       "      <td>8</td>\n",
       "      <td>16</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>light_apple</td>\n",
       "      <td>1013391</td>\n",
       "      <td>0.469441</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2020-03-01 08:16:09 UTC</td>\n",
       "      <td>cart</td>\n",
       "      <td>1004996</td>\n",
       "      <td>doogee</td>\n",
       "      <td>96.89</td>\n",
       "      <td>607174356</td>\n",
       "      <td>80d6850c-7f95-4978-ba1a-dedbe802e012</td>\n",
       "      <td>0</td>\n",
       "      <td>construction</td>\n",
       "      <td>tools</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-01 08:16:09</td>\n",
       "      <td>8</td>\n",
       "      <td>16</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>light_doogee</td>\n",
       "      <td>769</td>\n",
       "      <td>0.405722</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2020-03-01 08:16:09 UTC</td>\n",
       "      <td>cart</td>\n",
       "      <td>1005135</td>\n",
       "      <td>apple</td>\n",
       "      <td>1516.10</td>\n",
       "      <td>620967403</td>\n",
       "      <td>2f69a6e0-3a9e-4b7c-b717-ce5b8ad85ce3</td>\n",
       "      <td>0</td>\n",
       "      <td>construction</td>\n",
       "      <td>tools</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-01 08:16:09</td>\n",
       "      <td>8</td>\n",
       "      <td>16</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>light_apple</td>\n",
       "      <td>1013391</td>\n",
       "      <td>0.469441</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2020-03-01 08:16:13 UTC</td>\n",
       "      <td>cart</td>\n",
       "      <td>1005256</td>\n",
       "      <td>xiaomi</td>\n",
       "      <td>141.29</td>\n",
       "      <td>571788375</td>\n",
       "      <td>da050faa-118a-405a-b9c8-63f9d730328e</td>\n",
       "      <td>0</td>\n",
       "      <td>construction</td>\n",
       "      <td>tools</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-01 08:16:13</td>\n",
       "      <td>8</td>\n",
       "      <td>16</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>light_xiaomi</td>\n",
       "      <td>510657</td>\n",
       "      <td>0.396346</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461714</th>\n",
       "      <td>2020-03-31 19:25:14 UTC</td>\n",
       "      <td>purchase</td>\n",
       "      <td>18301044</td>\n",
       "      <td>&lt;NA&gt;</td>\n",
       "      <td>11.04</td>\n",
       "      <td>572119027</td>\n",
       "      <td>172b36e9-9259-423c-bc43-5d555ff94ce4</td>\n",
       "      <td>1</td>\n",
       "      <td>sport</td>\n",
       "      <td>ski</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-31 19:25:14</td>\n",
       "      <td>19</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>31</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>&lt;NA&gt;</td>\n",
       "      <td>&lt;NA&gt;</td>\n",
       "      <td>0.366924</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461715</th>\n",
       "      <td>2020-03-31 19:25:17 UTC</td>\n",
       "      <td>purchase</td>\n",
       "      <td>100058915</td>\n",
       "      <td>iqos</td>\n",
       "      <td>43.76</td>\n",
       "      <td>620477097</td>\n",
       "      <td>47786b4a-f2c3-48fa-b714-9d05556d5b98</td>\n",
       "      <td>1</td>\n",
       "      <td>apparel</td>\n",
       "      <td>trousers</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-31 19:25:17</td>\n",
       "      <td>19</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>31</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>&lt;NA&gt;</td>\n",
       "      <td>&lt;NA&gt;</td>\n",
       "      <td>0.366924</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461716</th>\n",
       "      <td>2020-03-31 19:25:36 UTC</td>\n",
       "      <td>purchase</td>\n",
       "      <td>32401283</td>\n",
       "      <td>&lt;NA&gt;</td>\n",
       "      <td>22.97</td>\n",
       "      <td>635102002</td>\n",
       "      <td>d82b8bf0-dea5-4e53-84f8-e61332eb17f1</td>\n",
       "      <td>1</td>\n",
       "      <td>apparel</td>\n",
       "      <td>underwear</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-31 19:25:36</td>\n",
       "      <td>19</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>31</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>&lt;NA&gt;</td>\n",
       "      <td>&lt;NA&gt;</td>\n",
       "      <td>0.366924</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461717</th>\n",
       "      <td>2020-03-31 19:26:18 UTC</td>\n",
       "      <td>purchase</td>\n",
       "      <td>4800282</td>\n",
       "      <td>samsung</td>\n",
       "      <td>38.59</td>\n",
       "      <td>622434648</td>\n",
       "      <td>4894c1b9-d00d-4418-b6c9-e8cd2f842b33</td>\n",
       "      <td>1</td>\n",
       "      <td>sport</td>\n",
       "      <td>bicycle</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-31 19:26:18</td>\n",
       "      <td>19</td>\n",
       "      <td>26</td>\n",
       "      <td>1</td>\n",
       "      <td>31</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>&lt;NA&gt;</td>\n",
       "      <td>&lt;NA&gt;</td>\n",
       "      <td>0.366924</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461718</th>\n",
       "      <td>2020-03-31 19:26:20 UTC</td>\n",
       "      <td>purchase</td>\n",
       "      <td>100058915</td>\n",
       "      <td>iqos</td>\n",
       "      <td>43.76</td>\n",
       "      <td>620477097</td>\n",
       "      <td>47786b4a-f2c3-48fa-b714-9d05556d5b98</td>\n",
       "      <td>1</td>\n",
       "      <td>apparel</td>\n",
       "      <td>trousers</td>\n",
       "      <td>...</td>\n",
       "      <td>2020-03-31 19:26:20</td>\n",
       "      <td>19</td>\n",
       "      <td>26</td>\n",
       "      <td>1</td>\n",
       "      <td>31</td>\n",
       "      <td>3</td>\n",
       "      <td>2020</td>\n",
       "      <td>&lt;NA&gt;</td>\n",
       "      <td>&lt;NA&gt;</td>\n",
       "      <td>0.366924</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2461719 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                      event_time event_type  product_id    brand    price  \\\n",
       "0        2020-03-01 08:16:04 UTC       cart     1005135    apple  1516.10   \n",
       "1        2020-03-01 08:16:08 UTC       cart     1005135    apple  1516.10   \n",
       "2        2020-03-01 08:16:09 UTC       cart     1004996   doogee    96.89   \n",
       "3        2020-03-01 08:16:09 UTC       cart     1005135    apple  1516.10   \n",
       "4        2020-03-01 08:16:13 UTC       cart     1005256   xiaomi   141.29   \n",
       "...                          ...        ...         ...      ...      ...   \n",
       "2461714  2020-03-31 19:25:14 UTC   purchase    18301044     <NA>    11.04   \n",
       "2461715  2020-03-31 19:25:17 UTC   purchase   100058915     iqos    43.76   \n",
       "2461716  2020-03-31 19:25:36 UTC   purchase    32401283     <NA>    22.97   \n",
       "2461717  2020-03-31 19:26:18 UTC   purchase     4800282  samsung    38.59   \n",
       "2461718  2020-03-31 19:26:20 UTC   purchase   100058915     iqos    43.76   \n",
       "\n",
       "           user_id                          user_session  target  \\\n",
       "0        620967403  2f69a6e0-3a9e-4b7c-b717-ce5b8ad85ce3       0   \n",
       "1        620967403  2f69a6e0-3a9e-4b7c-b717-ce5b8ad85ce3       0   \n",
       "2        607174356  80d6850c-7f95-4978-ba1a-dedbe802e012       0   \n",
       "3        620967403  2f69a6e0-3a9e-4b7c-b717-ce5b8ad85ce3       0   \n",
       "4        571788375  da050faa-118a-405a-b9c8-63f9d730328e       0   \n",
       "...            ...                                   ...     ...   \n",
       "2461714  572119027  172b36e9-9259-423c-bc43-5d555ff94ce4       1   \n",
       "2461715  620477097  47786b4a-f2c3-48fa-b714-9d05556d5b98       1   \n",
       "2461716  635102002  d82b8bf0-dea5-4e53-84f8-e61332eb17f1       1   \n",
       "2461717  622434648  4894c1b9-d00d-4418-b6c9-e8cd2f842b33       1   \n",
       "2461718  620477097  47786b4a-f2c3-48fa-b714-9d05556d5b98       1   \n",
       "\n",
       "                cat_0      cat_1  ...            timestamp ts_hour  ts_minute  \\\n",
       "0        construction      tools  ...  2020-03-01 08:16:04       8         16   \n",
       "1        construction      tools  ...  2020-03-01 08:16:08       8         16   \n",
       "2        construction      tools  ...  2020-03-01 08:16:09       8         16   \n",
       "3        construction      tools  ...  2020-03-01 08:16:09       8         16   \n",
       "4        construction      tools  ...  2020-03-01 08:16:13       8         16   \n",
       "...               ...        ...  ...                  ...     ...        ...   \n",
       "2461714         sport        ski  ...  2020-03-31 19:25:14      19         25   \n",
       "2461715       apparel   trousers  ...  2020-03-31 19:25:17      19         25   \n",
       "2461716       apparel  underwear  ...  2020-03-31 19:25:36      19         25   \n",
       "2461717         sport    bicycle  ...  2020-03-31 19:26:18      19         26   \n",
       "2461718       apparel   trousers  ...  2020-03-31 19:26:20      19         26   \n",
       "\n",
       "         ts_weekday  ts_day  ts_month  ts_year   cat_2_brand TE_count  \\\n",
       "0                 6       1         3     2020   light_apple  1013391   \n",
       "1                 6       1         3     2020   light_apple  1013391   \n",
       "2                 6       1         3     2020  light_doogee      769   \n",
       "3                 6       1         3     2020   light_apple  1013391   \n",
       "4                 6       1         3     2020  light_xiaomi   510657   \n",
       "...             ...     ...       ...      ...           ...      ...   \n",
       "2461714           1      31         3     2020          <NA>     <NA>   \n",
       "2461715           1      31         3     2020          <NA>     <NA>   \n",
       "2461716           1      31         3     2020          <NA>     <NA>   \n",
       "2461717           1      31         3     2020          <NA>     <NA>   \n",
       "2461718           1      31         3     2020          <NA>     <NA>   \n",
       "\n",
       "          TE_mean  \n",
       "0        0.469441  \n",
       "1        0.469441  \n",
       "2        0.405722  \n",
       "3        0.469441  \n",
       "4        0.396346  \n",
       "...           ...  \n",
       "2461714  0.366924  \n",
       "2461715  0.366924  \n",
       "2461716  0.366924  \n",
       "2461717  0.366924  \n",
       "2461718  0.366924  \n",
       "\n",
       "[2461719 rows x 22 columns]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "ddf_train.compute()\n",
    "ddf_valid.compute()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mon Sep 21 14:10:50 2020       \n",
      "+-----------------------------------------------------------------------------+\n",
      "| NVIDIA-SMI 440.64.00    Driver Version: 440.64.00    CUDA Version: 10.2     |\n",
      "|-------------------------------+----------------------+----------------------+\n",
      "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
      "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
      "|===============================+======================+======================|\n",
      "|   0  Tesla T4            Off  | 00000000:00:1E.0 Off |                    0 |\n",
      "| N/A   36C    P0    33W /  70W |   1629MiB / 15109MiB |      0%      Default |\n",
      "+-------------------------------+----------------------+----------------------+\n",
      "                                                                               \n",
      "+-----------------------------------------------------------------------------+\n",
      "| Processes:                                                       GPU Memory |\n",
      "|  GPU       PID   Type   Process name                             Usage      |\n",
      "|=============================================================================|\n",
      "+-----------------------------------------------------------------------------+\n"
     ]
    }
   ],
   "source": [
    "!nvidia-smi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "client.close()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
